Six Years of Systematic Literature Reviews in Software Engineering: an Extended Tertiary Study

After the introduction of evidence-based software engineering in 2004, systematic literature reviews (SLR) have been increasingly used as a method for conducting secondary studies in software engineering. Our goal is to analyze quality, coverage of software engineering topics, and potential impact of published SLRs, extending and updating two previous similar studies. We searched for SLRs published between July 2008 and December 2009, analyzed the relevant studies, compared and integrated our findings with previous studies. We found 67 SLRs, addressing 24 software engineering topics. The number of SLRs in software engineering is increasing and the overall quality of the studies is improving, suggesting that the software engineering research community is starting to consistently adopt SLRs as a research method. However, the majority of the SLRs did not evaluate the quality of primary studies and fail to provide guidelines to practitioners, thus decreasing their potential impact to influence software engineering practice.